Descent Direction Stochastic Approximation Algorithm with Adaptive Step Sizes
نویسندگان
چکیده
منابع مشابه
Non-parametric Stochastic Approximation with Large Step sizes
HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau...
متن کاملA stochastic approximation algorithm with multiplicative step size adaptation
An algorithm of searching a zero of an unknown function φ : R → R is considered, xt = xt−1−γt−1yt, t = 1, 2, . . ., where yt = φ(xt−1)+ξt is the value of φmeasured at xt−1 with some error, ξt is this error. The step sizes γt > 0 are random positive values and are calculated according to the rule: γt = min{u γt−1, ḡ} if yt−1yt > 0, and γt = d γt−1, otherwise. Here 0 < d < 1 < u, ḡ > 0. The funct...
متن کاملA stochastic approximation algorithm with multiplicative step size modification
An algorithm of searching a zero of an unknown function φ : R → R is considered: xt = xt−1 − γt−1yt, t = 1, 2, . . ., where yt = φ(xt−1) + ξt is the value of φ measured at xt−1 and ξt is the measurement error. The step sizes γt > 0 are modified in the course of the algorithm according to the rule: γt = min{u γt−1, ḡ} if yt−1yt > 0, and γt = d γt−1, otherwise, where 0 < d < 1 < u, ḡ > 0. That is...
متن کاملSwitched-mode systems: gradient-descent algorithms with Armijo step sizes
This paper concerns optimal mode-scheduling in autonomous switched-mode hybrid dynamical systems, where the objective is to minimize a cost-performance functional defined on the state trajectory as a function of the schedule of modes. The controlled variable, namely the modes’ schedule, consists of the sequence of modes and the switchover times between them. We propose a gradient-descent algori...
متن کاملOnline Learning with Adaptive Local Step Sizes
Almeida et al. have recently proposed online algorithms for local step size adaptation in nonlinear systems trained by gradient descent. Here we develop an alternative to their approach by extending Sutton’s work on linear systems to the general, nonlinear case. The resulting algorithms are computationally little more expensive than other acceleration techniques, do not assume statistical indep...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of Computational Mathematics
سال: 2019
ISSN: 0254-9409,1991-7139
DOI: 10.4208/jcm.1710-m2017-0021